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Let {X„}„>o be a ^-geometrically ergodic Markov chain. Given some real- valued functional F, 
define M„(a) := X]j?=i F{a,Xh-i,Xh), a € ^ C R. Consider an M estimator 3„, that is, 
a measurable function of the observations satisfying M„(a„) < miuag^ M„(a) -|-c„ with {c„}„>i 
some sequence of real numbers going to zero. Under some standard regularity and moment 
assumptions, close to those of the i.i.d. case, the estimator S„ satisfies a Berry-Esseen theorem 
uniformly with respect to the underlying probability distribution of the Markov chain. 
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1. Introduction 

Let {E,£) be a measurable space with £ a countably generated cr-field, and let {X„}„>o 
be a Markov chain with state space E and transition kernels {Qg{x,-): x £ E}, where 
is a parameter in some general set Q. The initial distribution of the chain, that is, 
the probability distribution of Xq, is denoted by fi and may or may not depend on 9. 
Although {Xn}n>o does not need to be the canonical version, we use the standard nota- 
tion Fg^fj, to refer to the probability distribution of {^n}ri>o (and Eg_^ for the expectation 
w.r.t. Pe.^i). We consider that {Xn}n>o is a ^-geometrically ergodic Markov chain, where 
V : E-^[l,+oo) is some fixed unbounded function. This class of Markov chains is large 
enough to cover interesting applications (see [16], Sections 16.4 and 16.5). 

The parameter of interest is ao = cto{9) C A, where ao(') is a function of the parame- 
ter 9 and A is an open interval of K. To estimate ao, let us introduce the statistic 

1 " 

Mn{a):^-Y,F{a,Xk^uXk), (1) 
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where is a real- valued measurable functional on Ax E^. We define an M -estimator 
(this is slightly more general than the usual definition of M-estimators or minimum 
contrast estimators, where c„ = 0, see [1]) to be a random variable a„ depending on the 
observations {Xq, . . . , Xn) such that 

Mn{a„) < minM„(a) + c„, 

where {c„}„>i is a sequence of non-negative real numbers going to zero to be specified 
later. Assume that for all 6 G Q 

Me{a) := lim E(,.,jAf„(a)] 

n— >oo 

is well defined everywhere on A and does not depend on ^. In addition, assume that there 
exists a unique "true" value of the parameter of interest, that is, Me(ao) < Mg{a), 
Va ^ ao . We want to prove the following uniform Berry-Esseen bound for a„ 



sup sup 



t{0) 



(S„ -ao)<u \ - r(u) 



01^ 

In 



(BE) 



where F denotes the standard normal distribution function, and t(0) is some positive 
real number defined in Theorem 3. 

To derive (BE), we use Pfanzagl's approach [20]. Besides technical assumptions, this 
approach relies on several ingredients. First, we need the uniform consistency condition: 

(UC) Vd>0, supegePe,M{l«"-ao| >4 = 0(1/V^). 

Second, consider the following two convergence properties: If iS'„(ao) •= X]fc=i 'C(q^Oi 
Xfe_i,Xfc) with f(ao,Xfc_i,Xfc) centered, 

(a) the sequence {E6/,;^[S'^(ao)]/n}„>i converges to a real number a"^ (9); 

(b) there exists a positive constant B{^) such that for any n>l 



sup sup 



-T{u) 



< 



The properties (a) and (b) will be required for certain ^(ao, y) defined as linear combi- 
nations of some functionals related to F. To obtain (a) and (b) for such ^(ao,a;,i/) with 
^-geometrically ergodic Markov chains, a natural moment (or V^-domination) condition 
is used: There exist positive constants and m such that 



y{x,y) e E'yaeA |e(a,x,2/)r < C^iV{x) + V{y)). 



(2) 



The paper is organized as follows. In Section 2, an extended version of Pfanzagl's theo- 
rem [20], is stated for any sequence of observations, not necessarily Markovian. Section 3 
is devoted to a Berry-Esseen bound for the additive functional J2^=i^i(^o,Xk-i,Xk) 
of a F-geometrically ergodic Markov chain {Xn}n>o with ^ satisfying inequality (2). In 
Section 3.2, we prove that the properties (a) and (b) are fulfilled when inequality (2) 
holds with the (almost expected) order m, namely: m > 2 ^ (a), and m > 3 (b). 
These results follow from the weak spectral method based on the theorem of Keller and 
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Liverani [14]. This approach, mtroduced m [10], is fuhy described in [12] in the Markov 
context (see also [8, 9] and other references given in [12]). It is important to notice that 
Pfanzagl's method requires the precise control of the constant B{^) in property (b) as 
a function of the size of ^. The present operator-type approach shows that B{£) de- 
pends only on the constant in inequahty (2). Thanks to these prchminary results, in 
Section 4 we prove our main statement, that is: 

(R) Under some technical assumptions and the uniform consistency condition (UC), 
if two functionals F' and F" related to F (in the basic case F' and F" are the 
first- and second-order derivatives of F with respect to a) satisfy inequality (2) for 
some m > 3 and constants Cp' , Cp" that do not depend on a, then din satisfies 
property (BE). 

To the best of our knowledge, the result (R) is new. It completes the central limit the- 
orem for proved in [5] when inequality (2) holds with m — 2. The domination 
condition (2) required by (R) is almost optimal in the sense that we impose to > 3 in 
place of the best possible value to = 3 obtained in the i.i.d. case. In Section 5, our results 
are applied to the AR(1) process with ARCH (autoregressive conditional heteroscedastic) 
of order-1 errors. The paper ends with a conclusion section. 

Let us close the Introduction with a brief review of previous related works in the 
literature. In [20], {X„}„gN is a sequence of i.i.d. random variables and Pfanzagl proved 
a Berry-Esseen theorem for minimum contrast estimators (which are special instances 
of M-estimators) associated with functionals of the form F{a,Xk)- In [20], the moment 
conditions on F' :— dF/da, F" := d^F jdo? are the expected ones since the property (b) 
is fulfilled under the expected third moment condition [6], Chapter XVI. Using convexity 
arguments, Bentkus et al. [2] proposed an alternative method for deriving Berry-Esseen 
bounds for M-cstimators with i.i.d. data. In the Markov context, the method proposed 
by Pfanzagl is extended, first by Rao to cover the case of uniformly ergodic Markov 
chains [21], second in [19] to the case of the linear autoregressive model. However, their 
assumptions to get the property (BE) include much stronger moment conditions involving 
both the functional F and the Markov chain. Here, as already mentioned, the weak 
spectral method of [12] enables us to have an (almost) optimal treatment of (a) and (b), 
and hence an improved Berry-Esseen result (BE). 

2. The Pfanzagl method revisited 

We state and prove a general result that allows us to derive uniform Berry-Esseen bounds 
for M-estimators. This result is an extended version of Theorem 1 in [20] and is applied 
to our Markov context in Section 4. 

2.1. The result 

Consider a statistical model (O, T ^ {Pq, d E O}), where 8 denotes some parameter space, 
and let {Xn}n>o be any sequence of observations (not necessarily Markovian). Let us 
denote the expectation with respect to Fq by Eg. 
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For each n, let M„(a) be a measurable functional of the observations Xq, . . . ,Xn and 
the parameter of interest a G A, where A is some open interval of K. Let {c„}„>i be 
a sequence of non-negative real numbers going to zero at some rate to be specified later. 
An M -estimator is a measurable function a„ of the observations (^Oj • • • j Xn) such that 



Mn(S„) < minM„(Q!) + c„ 



(3) 



This is the usual definition of minimum contrast estimators as soon as c„ = . 

Assumptions. Suppose that for all n> 1 and a Cz A, there exist M^{a), M^(a) some 
measurable functions depending on Xq,Xi, . . . , Xn and on the parameter of interest, such 
that the following properties hold true: 

(Al) yO Cz 0, there exists a unique ao = c(q(9) € A such that Mg{ao) = 0, where 
M'g{a) 

[e,a)(z<dxA); 



lim„_j.oo IEe[Af,'j(Q;)] (the limit is assumed to be well defined for all 



lim,. 



,Ee[Af;'(ao)] (the 



(A2) < infgge '7?(6') < supggQm(0) < oo, where m{6) 

limit is assumed to be well defined for all 9 ); 
(A3) for every n>\, there exists r„ > independent of 9 such that r„ ~ o(n~^/^) and 

supP4|M;(a„)| > r„} = 0{n-^'^)- 
eee 

(A4) for j = 1,2, there exists a function (Jj() such that < infgge < 
supggQ o'jid) < oo and there exists a positive constant B such that for all n>l 



sup sup 



a,i9) 



M^iao)<u\~riu) 



sup sup 



{M::{ao)~mi9))<u\^Tiu) 



< 



< 



B 

B 



(A4') for n> 1, \u\ < 2\/lnn and 9 £ Q, there is a positive number (T„.„(0) such that 

|a„.„(0)-ai(0)| <A'M, 



M;K) + ^iM_(M:(ao) - m{9)) ] < u 



T{u) 



< 



with some positive constants A' , B' independent ofn,u,9; 
(A5) for any (a, a) G A^ , let Rn{a,a) be defined by the equation 

M'^[a) = M'^ia) + [A/;'(a) + i?„(a, a)] (a - a). 

For each n, there exist u;„ > and a real-valued measurable function Wn de- 
pending on Xq, . . . , X„, both independent of 9, such that aj„ = o(l) and 



V(a,a) e A^ 



\Rnia,a)\<{\c 
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and there is a constant cw > such that 

supPe{cH/< t¥„} = 0(n-i/2); 
See 

(A6) a„ is assumed to be uniformly consistent, that is, there exists j„ =o(l) such 
that 

supPe{|3„ - aol > d} < 7„, 

where d := mig^Qm{9)/8cw with cw cind m{6) defined in (A5) and (A2), re- 
spectively. 

Let us comment on these assumptions. Condition (Al) identifies the true value of the 
parameter. In conditions (Al) and (A2), the expectations E6/[M^(a)] and Ee[M"(ao)] 
may depend on n, as in the Markovian framework considered in the sequel when the 
initial distribution is not the stationary distribution. Condition (A3) ensures that the es- 
timator (approximately) satisfies a kind of first-order condition. Such a condition allows 
us to take into account the numerical errors with which we are faced when computing a„. 
It may also be useful when the estimator of the parameter ao depends on some "nui- 
sance" parameters (see the example in the second part of Section 5). Conditions (A4) 
and (A4') are the uniform Berry-Esseen bounds for M^(ao), M"(ao) and for some of 
their linear combinations. The identity defining i?„(a, a) in condition (A5) is guaranteed 
by a Taylor expansion when the criterion M„(a) is twice differentiable with respect to a. 
In this case M'^ and M^' are nothing else but the first- and second-order derivatives 
of Mn with respect to a. The reminder Rn{a,a) must satisfy a Lipschitz condition. For 
instance, when a;„ ~ 0, this holds true if a n- M„(a) is three times continuously differen- 
tiable with a bounded third-order derivative. Condition (A6) is a standard consistency 
condition (see [2]). General sufficient conditions for (A6) with 7„ = 0(n~^) have been 
proposed in the case of i.i.d. observations or uniformly ergodic Markov chains (see [18], 
Lemma 4, and [21], Lemma 4.1, resp.). Such general arguments can easily be adapted 
to the geometrically ergodic Markov chain framework. In specific examples, like the one 
investigated in Section 5, condition (A6) can be checked by direct arguments. 

The proof of Theorem 1, which adapts the arguments of [20], is given in Section 2.2. 

Theorem 1. Under conditions (A1)-(A6), there exists a positive constant C such that 



Vn > 1 sup sup 



with t{9) ■.= (Ti{e)/m{e). 



{a„ - ao) <u\- - T{u) 



< C ( ^ + V^r„ + OJn + In ] (4) 



To obtain the classical order 0(n^^/^) of the Berry-Esseen bound, one needs 7„ = 
0(ri~^/^), r„ = 0{n~^) and aj„ = 0(n^^/^). Note that this usually requires that the se- 
quence {c„}„>i in (3) decreases at the rate n~^/'^. This is to be compared to the rate 
that is usually required to obtain the asymptotic normality of M-estimators (see [1]). 
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Remark 1. A close inspection of the proof of Theorem 1 below shows that the con- 
stant C in inequality (4) can be tracked provided that the O(-) and o(-) rates in as- 
sumptions (A3)-(A6) are more explicit. For the sake of brevity, we only consider the 
case where c„ = r„ = cj„ = 0, a{9) = 9 and (A3) is: for any n > 1, |M,j(0„)| = 0. The 
constants C in the various inequalities of assumptions (A4)~(A6) arc denoted by Ci,C2 
in (A4), C3,C4 in (A4') and C5 in (A5) and we choose 7„ < Cen"^/^ in (A6). Then we 
can obtain from Propositions 1 and 2 that 



Vn> 1 



sup 



(a„ - ao) <u \ - T{u) 



where C := 4 



1 

/2n 



2n 



2Ci 



2Ci -I- 
4C + 2 ""'^^"°''/^^ 



r{9) 

2r< 1 cxp(-a''/2) 

2C5 + Cg 



< 



c 



C5 4 
f C4 



Cg when |u| > 2vlnn; or 

_ 16c~^(C3+g^cw) 
cr, V27T 



when lul < 



2\/\nn provided that y^n/lnn > ma,x{8cw cr^ , 4) / a^; with a := mfg,=Q{Tn{9)/4:a2{9)), 
:= supgg0(Ti(6')/m(6'), infgge o-iC^*)- 



2.2. Proof of Theorem 1 



The hypotheses of Theorem 1 are assumed to hold. For the sake of brevity, the sequence 
{rn}n>i in (A3) is supposed to be such that r„ = o(n^^/^) and |Af4(a„)| < r„ for every 
n > 1. In the general case, it suffices to work on the event {|M^(a„)| < r„} and to bound 
the probability of the event {|M^(a„)| > r„} using (A3). From conditions (A2) and (A4), 



r{9) 



MO) 
m{9) ' 

inf aA9), 
eee 



m :— inf m(9), 



sup m(9), 
eee 



sup aj {9), 
eee 



j = 1, 2, are well defined. Recall that < m < m < 00 and < <aj < 00. Note that 
the function t(-) is positive and bounded. In the following, C denotes a positive constant 
whose value may be different from line to line. 

Inequality (4) is proved, first for |u| > 2A/lnn, second for |w| < 2\/lnri. In fact, for 
\u\ > 2^/hin, the bound in inequality (4) does not involve r„ and w„. 

Proposition 1. There exists a positive constant C such that for each n>l and all u€M. 
such that \u\ > 2^/hrn 



sup 

eee 



r{9) 



(a„ - ao) < u > - r(it) 



C 



(5) 



Proof. For |u| > 2\/lnn, it is easily checked that 
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Now, 



r(-2VW) < 



1 1 

2^/\an \/27t J2 



+ 00 



1 1 1 



' In n 



2Vliin \/2nn 



Finally, the proof is complete if there exists C > such that (see [18], Lemma 6) 



Vn > 1 sup I 
see 



r{e) 



|a„ - aol > 2vhin \ < 



C 



In- 



(6) 



It follows from (A5) and (A3) that |M4(ao)| +rn> — oio\\M^{ao) + i?„(a„,ao) 
Then, 



C7l(0) 



n ,^ vlnn 
an- ao\>2 



MO) ^ :^(\^'^nM\+rn) > 2-^|Af;'(ao) +i?„(a„,ao)|, 



provided that M4(a„) 7^ M^(ao). Next, introducing the event {2|M4'(ao) + -Rn(Sn, 
ao)| > m{9)} and its complement (which includes the event {M4(a„) = M4(ao)}), we 
obtain 



|S„ — aol > 2vlnn 



t{9) 



< 



{|M;(ao)| + Tn} > Vh^ ^ +Pe{2|M;'(ao) + i?„(a„,ao)| < m{0)}- 



It is easily checked from (A4) and r„ = o(n ^/^) that 



supi 



C7l(0) 



{|M;(ao)|+'^n}>\/W^=0^ +2r -x/h^ 



^ =0 



Finally, to obtain the bound (6), it remains to justify the use of the following bound: 
supPe{2|A/;'(ao) + i?n(S„, ao)| < m{e)} = 0{n~^/^-) + 7„. 

Using elementary inequalities and assumption (A5), 

Pe{2|M;'(ao) + i?„(S„,ao)| < m(0)} 

< Pe{|Af;'(ao) - m{e)\ > m(0)/4} + ¥g{\Rni5„,ao)\ > m{e)/4} 
<Pe{|A/;'(ao) - m{e)\ > to(0)/4} + Pe{ [|S„ - ao)| + w„]W„ > m(0)/4} 
= : Pi,n,e + P2,n,e- 

It follows from (A4) that a := inteee {m.{6) /4:<T2{0)) is well defined and positive, and 

sup Pi,„,e < 0{n-^^^) + 2r(-aV^) = 0{n-/'^/^). 
9ee 



(7) 



(8) 
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Now, let d{9) ■.= m{9)/Acw with cw defined in (A5) and notice that d = infg>^0d{d')/2 
in (A6). Use the event {|a„ — ao| < d{0) — uj„} and its complement to write 



< 



r{9) 



< [|S„ -ao\+ uJn]Wn < Wnd{e) \ + Pe{|a„ - ao| > d{e) - w„} 



< snpPe{cw < Wn} + supPe{|5„ - ao| > 4 - 0(n-i/2) ^ 
eee eee 



from (A5)-(A6) and provided that cj„ < d. Therefore, inequality (7) holds true. 

Now, it remains to investigate the case \u\ < 2-\/lnn. 
Proposition 2. There exists a positive constant C such that, for any \u\ < 2\/\nn, 



□ 



sup 

eee 



-{6) 



(S„ -~ao)<u} - T{u) 



<C 



1 



(9) 



Proof. We just have to prove that (9) holds true for all n>no, for some no G N. Let us 
introduce some sets and derive their probability bounds: 

• E„M {V^|S„ - ao\/T{e) < 2^1^}. From (6), sup^ge Pe(-Ef,,(,) = 0{n-^/^ + -fn). 

• An {0 < Wn < Cw} where the r.v. Wn and the constant cw are defined in (A5). 
Then supegel'e(A^) = 0(n-i/2). 

. Dn,e := {2Af;:(ao) > m{e)}. We have n{D^^,e} < - m{e)\ > m(0)/2} < 

Fs{\M^{aa)-m{9)\ > m(6')/4}. We know from (8) that supg^QFg{D^g) = 0(7i-i/2). 

Then, we obtain from the previous estimates that the following set 

Bn,e ■— En^e n A„ n Dn^e 

is such that 

-In). (10) 

Now, if Dn,0,u ■= {\/n(Sn — ao)/T{6) < u}, then we can write from (10) 

|Pe(A.,e,«) - nu)\ < |Pe(A^,e,n n Bn,e) - r{u)\ + 0{n-'/^ + 7„). (11) 
From (A2) and (A4), < := supg^Q t(6') < oo. Define the piecewise quadratic functions 
{v) -.^ c^ + V + v'^ , 5+(t;) := c+ + + a+u^ (12) 

where := n[M^(ao) ± r„],6='= := T{9)y/n[Mn{ao) ± sign(v)cvi/i^n],a''' ■— io'^cvi/, and 
sign(w) denotes the sign of v when v ^ and sign(O) = 0. Notice that g~ and 5"*" are 
continuous on the whole real line. To bound the term \Ve{Dn,e,u H i?n,e) ^ r(u)| in (11), 
let us introduce the events 



supPe(B;i.e)<0(n 
eee 



-1/2 



■.= {g^{u)>0}. 



(13) 
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so that 



< max{|Pe(i?,;,,,„ n B„,g) ~ T{u)l |Pe«e,„ n B,,.e) - T{u)\} 

< me.x{\Pg{E-,J ~ r{u)\, \Fe{E+J - T{u)\} +Pg{B:^s). 



(14) 



Then the proof of Proposition 2 is easily completed using (10) and the following estimate: 
There exists a constant C such that for n large enough and |m| < 2-\/lnn 



1 



sup \¥eiE^g J - r(u)| <C[^ + x/^r„ + cj„ 
see ' ' Vv" 

Indeed, E^g ^ = {g^{u) > 0} with defined in (12). Wc can write 

En,e,u = {"[^^n(ao) ± r„] + uT{0)y/^[M',^{an) ± sign(u)cvyw„] ± u^a^cw > 0} 



(15) 



<(«o) + 



uai{e) 



(M:(ao)-m(0)) > 



a„(u,6') + bn{u,8) 



where the positive real number an,u{Q) is that of condition (A4') and 



an{u,9) = u 



sign(w)cvFW„ , , ua'^cw 
<^U^^)(1 ± 7^^ ) ± — -j=- 



b„{u,6) = ±^/n^„. 



From the second statement of (A4'), it follows that there exists a constant B' such that 
we have, for n large enough and |m| < 2-\/lnn, 



sup 



n-e'„,e,«) — 



an{u,9) + bniu,0) 

¥) 



< 



B' 



Now, from a-^ := infgge o'i(^) > and from the first property of (Tn.u{&) hi (A4'), it follows 
that, for n large enough and \u\ < 2\/\nn, and for all e 8, wc have cr„.„(6') > g_i/2 and 



an{u,9) 



< 



< 



2\u 



CW^^n \u\cF Cw 

m{9) ^/n 



, Ai 1 \ 1^1 

Jn m 



/ 2 






- \u\0Jn^ 







where C" is independent of n, u, 9. We obtain from estimates on the characteristic 
function of the standard Gaussian distribution reported in [20], page 89, that, for n large 
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enough, |u| < 2\/lnn, and 9 £0, 



Tin) 



1 



. a'„,„(e') 

for some Ci > 0. Wc deduce from similar arguments that, for some constant C2, 
( an{u,e) \ ( a„{u,e) + bn{u,e) \ _ 

\cr„,u(0) J V CTnA^) ) 

Since Ci, C2 only depend on A', CTi, m, ct and cvk, the proof of (15) is complete. 



□ 



3. A Berry— Esseen bound for an additive functional 
of geometrically ergodic Markov chains 

The main focus of the paper is to apply the general Berry-Esseen result of Theorem 1 to 
the case of M-estimators as defined in the Introduction when the observations come from 
a geometrically ergodic Markov chain. To check conditions (A4) and (A4') in Theorem 1, 
we need the next probabilistic results based on a recent version of the Berry-Esseen 
theorem derived by [12] in the geometrically ergodic Markov chain setting. 

3.1. The statistical model 

Let {E,£) be a measurable space with a countably generated cr- field £ and Q be some 
general parameter space. Let {Xn}n>o be a Markov chain with state space E, transition 
kernels {Q0{x, •), a; £ E}, 9 £ & and an initial distribution /i that may or may not depend 
on 9. 

Assumption (A4). Let V : E^[l, +00) be an unbounded function (independent of 9). 
For each 9 CzQ, there exists a Qg -invariant probability distribution, denoted by iTg, such 
that 

(VGl) bi:=supg^QTTe{V) <+oo. 

(VG2) For all 7 G (0, 1], there exist real numbers Kj <1 and > such that we have, 
for any 9 £ 0, n>l and x £ E, 

snp{\Q]^f{x)^TTe{f)\,f:E^C measurable, |/| < ^ } < C^K!;F(a;)'^. 

Throughout Section 3, we assume that 'p{V) := supggQ /i(X^) < 00. Notice that (VG2) 
with 7 = 1 implies the following property: For any measurable real-valued function / 
defined on E such that |/| < DV, for some constant D > 0, 



Vti > 1 sup |Ee.^[/(X„)] - TTg{f)\ < DCiK^-p{V). 



(16) 
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Moreover, conditions (VGl) and (VG2) imply that, for any 7 G (0, 1] and 9 G<d, Qg is 
y''' -geometrically ergodic, but it is worth noticing that the constants and do not 
depend on 9. In the following remark, the properties (VGl) and (VG2) are related to the 
so-called drift condition w.r.t. the function V for each Qg. 

Remark 2. Assume that for each 9 G Q,Qg is aperiodic and V'-irreducible w.r.t. a certain 
positive (T- finite measure on E (which may depend on 9). 

1. For 7 ~ 1 and any fixed 9, the properties (VG1)-(VG2) follow from the drift con- 
dition: QgV < gV + <^ls, with g < 1,<; > and some set S {S is the so-called small 
set) satisfying the minorization condition Qg{x,-) > ci>{-)ls{x), where c>0 and v is 
a probability measure concentrated on S (see [16], Theorem 16.0.1). In addition, the 
constants Ci and ni can be bounded by a quantity involving g, c, the measure v and 
the set S (see [17]). To obtain the uniformity in 9, it suffices to check that all these 
elements do not depend on 9. 

2. For any 7 G (0, 1], we have TTg{V^) < TTg{V) and thus condition (VGl) implies that 
snpg^QiTgiV'') < 00. Furthermore, under the drift condition, it follows from Jensen's 
inequality that QgV < g''V + (;''ls- Using [17], one obtains (VG2). 



3.2. A preliminary uniform Berry— Esseen statement 

Let ao = ao (9) G ^ be the parameter of interest for the statistical applications we have in 
mind (see condition (Al), page 4), where 9 is the parameter of the Markov chain model 
and A is an open interval of the real line. 

Let ^{a,x,y) be a real-valued measurable function defined on ^ x E^ such that the 
random variable ^(a, Xk-i, Xk) is (integrable and) centered with respect to the stationary 
distribution irg, that is, 

Ee,.«[e(ao,^o,^i)]=0, 

and let 

n 

Sn{o-) := ^5(a,Xfc_i,Xfe). 

k=l 

We investigate the following uniform Berry-Essecn property: 



sup sup 

geOueR 



'j{9)^ 



<u \ - T{u) 



= 



where cr^ (9) will be defined below as the asymptotic variance associated with the random 
variables £,{a,Xk-i,Xk). When {X„}„>o are i.i.d. and £,{a,Xk-i,Xk) = £,{a,Xk), this 
property follows from the Berry-Esseen theorem [6], provided that ^(a,Xo) has finite 
third-order moment, uniformly bounded in a, and a variance greater than some positive 
constant that does not depend on a. 

In our Markov framework, the following moment (or ^-domination) condition is natu- 
ral for the functional ^. In the sequel, this condition will be required for toq = 1,2 or 3. 
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Condition (Dmg). There exist real constants m > mo > 1 and > such that 



Va e A,y{x,y) e \^{a,x,y)r < + V{y)). 

This domination condition implies that 

Ee,.,[|C(«,Xo,Xi)r"]= J \aa,x,y)rQe{x,dy)d7Te{x) 
<C^{ne{V)+7TeiQeV))<(^, 
and since m > 1, observe that Eg^^^ Xq, Xi)|] < oo. 



(17) 



Proposition 3. Suppose that Assumption (Ai ) holds true and that ^ is centered and 
satisfies condition (-Di). Then, we have supgg0Sup„>j |Ee^^[S'„(ao)]| < oo. In particular, 
for each G 8, lim„Ee.^i[5'„(ao)/n] = 0. //, in addition, ^ satisfies condition {D2), then 
for each 9 Cz Q, the non-negative real number 



a^{9) :=hm 



Ee,^[S'„(ao)^ 



is well defined and does not depend on ji. Furthermore, the function cr^(-) is hounded 
on 0, and there exists a positive constant C, only depending on and]l(V), such that 



Wee 



Vn> 1, 



a\0) 



Ee,^[S'„(ao)^ 



< 



C 



Now, we are ready to state our uniform Berry-Esseen statement for S„{ao). 

Theorem 2. Let us assume that: 

1. Assumption (M) holds true; 

2. the functional ^ is centered and satisfies condition {D3); 

3. af,~Mgee^^iO)>0. 



Then, there exists a constant B{^) such that 



V71> 1 



sup sup 



<u} -r{u) 



< 



B{0 



Furthermore, the constant B{^) depends on the functional ^, but only through (Tq and the 
constant of condition (-D3). 



The fact that we look for a Berry-Esseen bound with a constant B{^) independent 
of 9 is natural given our main purpose, that is, to prove a uniform Berry-Esseen theorem 
for M-estimators. 
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There are several methods for deriving Berry-Esseen bound for the functionals of 
Markov chains (see [3, 13]). But to prove Proposition 3 and Theorem 2, we use the 
weak spectral method developed in [12]. (A Berry-Esseen theorem is established in [11] 
for sequences of the form {C(-'ffc)}fe>o under the conditions ^jl{V) < oo and < CV; 
however, the case of sequences of the form {£^{Xk-i,Xk)}k>o is not a direct corollary of 
this work since the Markov chain {{Xk-i,Xk)}k>o may not be geometrically ergodic.) 
This method allows us to control the constant B{^) as a function of for checking 
assumption (A4') of Theorem 1 (see the arguments following equation (32) in Section 4). 
This follows from the next key technical result. Although the proof of the Berry-Esseen 
theorem only requires Taylor expansions up to the order mo and Condition (£)„„) with 
mo = 3, for the purpose of possible further applications, Lemma 1 below is stated for any 
mo e N* . 

Lemma 1. IfS,is centered and satisfies Condition (Dmo) with mo G N*, then there exists 
/3 > such that 

V0ee,Vn>l,Vte[-/3,/3] E(,,^[e'*^"("°)] = Ae(t)"(l + ie(t))+re,„(0, (18) 

where Xd{-), Lg{-) and re^„(-) are some mo times continuously dijferentiable functions 
from [— /3,/3] into C satisfying Xg{0) ~ 1, Ag(0) — 0,Lg{0) = and 7'6/,n(0) ~ 0. Further- 
more, there exists p G (0,1) such that we have for £ = 0, . . . , tuq ■' 

Ge := sup{p-"\rl%{t)l \t\ < (3,0 ee,n>l} < ^. 
Finally, the constants (3, p, Gt and the following ones (for £ — 0, . . . , mo ), 

Ei := sup{|A^')(<)|, \t\ < € 9} < cx), 
F, -.^ sup{|4''(t)|, |t| < /3,^? e 6} < oo, 
depend on ^, but only through the constant C^ of Condition (Dmg). 

Lemma 1 is proved in Section 3.3. The definition of Lg{t) and rg^n{t) (see (25) and (26)) 
shows that the constants Fi and Gt also depend on JllV) (see Remark 3). Now Lemma 1 
allows us to derive Proposition 3 and Theorem 2. 

Proof of Proposition 3. Assume that ^ is centered and satisfies (i?mo) with mo S N*. 
Proceeding as in (17) and using (16), (VGl) and fliy) < oo, we obtain that 

supsupEg.p[|^(Q;o, ATfe-i, ATfc)!'"] < oo for some m > 7«o . (19) 
eeefc>i 

Now assume mo = 1, and let 0(i) := Ee^^[e'*'^"("°)], t e M. Then (/)'(0) = iE0^^[S'„(ao)], 
but Lemma 1 also gives (j)'{0) = L'g{0) + „(0). Hence supggQ sup„>]^ 
Fi + Gi. Next, assume mo = 2. From (19) we have Eg.p[S'„(Q;o)^] < oo, and thus we 
can write </)"(0) = -Ee,^[S'„(ao)^], and 0"(O) = nX^{0) + L^'(O) + rl^{0) by Lemma 1. 
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Thus we obtain \X'^{0) +EgJS„iao)^]/n\ < (|L';(0)| + |r^',„(0)|)/n < (^^2 + Ga)/^. Set 
(T^{0) := ~Ag (0). Then supggQ < E2 (by Lemma 1), and the proof is complete with 



C:=F2 + G2 



□ 



Proof of Theorem 2. RecaU that ^ is centered and satisfies condition (D3). To prove 
the result, we use Lemma 1 with niQ = 3 and we adapt the arguments of the i.i.d. case. 
Recall that cr^(6') = — Ag(0). According to the classical Berry-Esseen inequality (see [6]), 
we must prove that for some suitable positive constant c, supg^Q An{6) = 0(n~^/^), 
where 



An{0) := 



E[( 



>itS„(ao)/(<T(e)V^)l 



t 



At. 



For the moment, we just assume that < c < /Jctqi where /3 is the real number in Lemma 1. 
Notice that |t| < c implies |t/cr(0)| < /3 for all Q S 0. Using Lemma 1, we have 



Afi 



t 



dt 



dt 



At 



By a Taylor expansion, for all 6* G O and < c. 



A. 



aie) 



- 1 



where is defined in Lemma 1. Hereafter, set c min {f3ao,3<j^/2E3,V^}. From the 
last inequality, deduce that for any |w| < c 



Afi 



aie) 



< 1 



V 

y 



Therefore, for any t e M such that |t| < Cy/n, 

t 



<e-*V^ 



Let us write 
t 



A« 



-t-/(2n) 



fc=0 



i-fc-i 



(20) 



-fcr/(2n) 
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Notice that \\e{t/a{e)^)-eyi-p{~t'^ /2n)\ < {a + E3/6a^)\t/^\^ if a := sup|„|<J?/'(3)(„)| 
with ip{v) := 6exp(— 1)^/2). Moreover, 



n-l 

E 



A:=0 



t 



ri — fe— 1 n—1 
-kt^/2n 



where & := sup|.„|<cexp(w^/4). Hence 
t 



e/2 



k=0 



< 



which yields supgge/„(6') < 6n~^/2(a + E's/Gcrg) J^t^ cxp(-tV4) dt. Next, using (20) and 



sup J„ (6*) < 
see CToV'T' 



Finally, using re^„(0) = 0, we have supg^e |re^„(t/<T(6')yn)| < (|t|/(To%Ai)Gip", so that 
supggQ if„(6') < (2cG'i/(To)p"- Gathering the results, we deduce that 

supA.(e)<^ + ^p", 

where the constants A, p, Gi and c depend on of condition {D^). The Berry-Esseen 
inequality [6] then yields 



sup 



Sn{e) 



\=<u 



T{u) 



2cGi 

/ 



24?7\ 



where rj = sup^^jj |r'(w)|. The proof of Theorem 2 is complete. 



□ 



3.3. Proof of Lemma 1 

For 9 G Q fixed, Lemma 1 follows from [12], Section 10. Here, we must prove that all the 
constants in Lemma 1 are uniform in 9 and depend on ^ as claimed. For this purpose, the 
weak spectral method is outlined below (in the X^-geometrical crgodicity context) and we 
give the main statements by paying special attention to the constants. For convenience, 
the technical proofs are postponed in Appendix B. 

• Geometrical ergodicity of Qg. Let < 7 < 1. We denote by B-^ the weighted 
supremum-normcd space of measurable complex- valued functions f on E such that 



/II7 ■= sup 



xeE 



1/(^)1 



< 00. 



(B-y, II • ll-y) is a Banach space. The space of bounded operators on is denoted by C{B^), 
and the associated operator norm is still denoted by || • ||^. We have from (VGl) 



sup7re(F'') < 61 = sup7r£/(F) < 00, 
eee fee 



(21) 
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so that TTg is a continuous linear form on B-y. Define tlie following rank-one projection 
on B^: 

yfeB^ ngf:=7Tg{fnE- 

Then condition (VG2) in Assumption (Ai) can be rewritten as follows: Qg e ^{Bj) and 
there exist < 1 and > such that 

yeee,vfeB^,Vn>i \\Q"gf -Ugf\\^<c^>,;\\f\\^. (22) 

From (21) and (22), HQ'^'H^ = sup^^j^{Q'f^V''){x)/V{xy' is uniformly bounded in n G N* 
and 0ee. 

• The Fourier kernels associated with Qg and ^. Assume that, for all a Cz 
^(a, •,•) is measurable. The Fourier kernels associated with Qg and ^ are denoted by 
{Qg(t)lx,dy),t€ R} and defined by 

yxeE Qg{t){x, dy) e'*«("°'"'^)g(,(a;, dy). 

Let us recall that S'„(ao) '■— ^(Q!o7 -^fc-i, -'ffe)- The following link between Qg{t) and 

the characteristic function of S'„(ao) is well-known in the spectral method: 

Vn>l,VteM Ee,^[e'*^"("«)]=/i(Qe(i)"lij). (23) 

In fact, we have Ee.^[e'*'^"^"''-'/(X„)] = fJ,{Qg{ty^ f) for any real-valued measurable 
bounded function f on E. This can be easily checked by induction using the Markov 
property and the following equality: 

Vn>2 Ee,^[e'*^"("'')/(X„)]=Ee,^[e'*^"-^("°)(Qe(<)/)(^n-i)]- 

• Spectral study of Qg{t) on B-y (for t near 0). It can be easily seen that, for all i e M, 
we have Qgit) € C{B.y). For n S (0, 1), we set 

I?„:={zeC: |z| > K, |z- 1| > (1 - k)/2}. 

Lemma 2. Let 7 S (0, 1). For all k G {n-y, 1), there exists (3^^^ > such that, for 9 Cz Q, 
\t\ < f3i,n o.nd z e X>„, we have (z — Qg[t))^^ e ^i^-y) 

7^^,« := sup{||(z - Qg{t)y\\y: OeO, \t\ < l3^^,,ze V,} < c3o. 

Moreover, the constants (3^^^ and TZy^^ depend on ^, but only via the constant of 
Condition (Dmo)- 

For fixed. Lemma 2 is established in [12], Proposition 10.1, thanks to the theorem 
of Keller and Liverani [14, 15]. Here, we only have to prove that the constants /3^^„ 
and TZy^K are uniform in 9 and depend on ^ as stated above. According to [14], Remark, 
page 145, it is enough to check that the constants are so involved in the hypotheses of 
the Keller-Liverani theorem. This is due to Lemmas B.1-B.2 in Appendix B. 
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• Proof of formula (18). Now assume that ^ satisfies Condition {D„i„) for some 
mo € N*. Let 70 € (0,1) be fixed such that 70 + mo/m < 1. For any k e (k^o,1), de- 
note by Fq^k the oriented circle centered at z = 0, with radius k, and by Fi.k the oriented 
circle centered at z = 1, with radius (1 — k)/2. Note that both Fq^k and Fi^^ are contained 
in From (22) and Lemma 2, one can deduce that we have, for all n > 1, 6* € 0. and 
t e [-/37o,k;^7o,k]i the following equality in C{By„): 

Qe {tr = Xe {trile (t) + Ne (t)" , (24) 

where Xe{t) is the dominating simple eigenvalue of Qe{t) and Ile{t) and Ng{t)" are the 
elements of £{B^„) defined by the following line integrals: 

ne(t):=^j( iz-Qe{t)r'dz and Ne{tY' -.^ j z'\z - Qe{t)y^ Az. 

Note that we have \e{Q) ~ 1 and 11^(0) = lie from (22). Also observe that, from Lemma 2 
and the definition of Fo,k, we have || A^6i(i)"||-y = 0(k"). Since Ib £ and Ijl{V) < 00 
(/X is a continuous linear form on B-yg), the equalities (23) and (24) give: 

Ee,^[e"^"("°)] = Ae(t)>(ne(t)l£) + M(7Ve(t)"l£). 
Therefore, formula (18) holds true with 

Lg{t) := tiineit)lE) - I, reAt)-=KMiT^E) (nGN*). 

We have Le{0) = A'(nelB) - 1 = and rg,,{{)) = ^(iVe(0)"l£) - ^^(g^lB - Heli;) = 0. 
Finally, to make the link with Lemma 3 below easier, let us observe that 

l + ie(i) = ^jj^ ^J^{{z-Qe{t)yhE)dz, (25) 

re,„(i) = ^j( z■^^i{{z~Qe{t))-hE)Az. (26) 

• Regularity properties of A(-), Lo{-), r'e.„(-). Let 79 be such that 70 + mo/m < 7q < 1. 
We denote by C{By„,BYj the space of the bounded linear operators from B^^ to B^i^, 
and by II • H^o.^i'i the associated operator norm. 

Lemma 3. We have the following regularity properties: 

(a) The map Qe{') is m^-times continuously differ entiahle from R to C{B-yg,Byi^) , and 

we have := sup^gR^gge Il<3e^^(0ll7o,7i < 00 /or £ = 0, ttiq. 

(b) There exist some real numbers k g (K-y„,l) and < /3 < /S-yo,^ such that, for all 
9 E and z G T)^, the function Rg^z : 1 1— > (z — Qg{t))~^ is m^-times continuously differ- 
entiable from [— into C(Bjg,B-y^), and we have for £ — 0, ... ,mo: 

sup{||<i(t)||^„,^,: |t| < /3,z e e 9} < 00. 
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The scalars (3, k and all the bounds in (a) and (b) depend on ^ only via the constant C'^ 
of Condition (Dm^). 

For 6 fixed, Lemma 3 is established in [12], Proposition 10.3. It can be also derived 
from [8], wliich relaxes the assumptions used in [9, 10] to obtain Taylor expansions of 
the resolvent maps. (As observed in [8], the passage to the differentiability properties 
can be derived from [4].) However, a fine control of the constants is still required. Using 
cither [8] or [12], Section 10, this control is derived from Lemma 2 and from Lemma B.3 
in Appendix B. 

Since Ig S B^^ and /U is a continuous linear form on B^'^ (use Jiiy) < cxd). Lemma 3(b) 
gives that, for any z £ F^k U Fi^^, the C- valued function t ^ fi{{z — Qg{t))~^lE) is 
mo-times continuously differentiable on [— /3,/3] and that its niQ first derivatives are uni- 
formly bounded in 9 and z € Fq.k U Tq,^. The regularity properties (and the related 
bounds) for Lg{-) and ?'e,n(') then follow from (25) and (26), while those concerning the 
function Xg{-) follow from both Lemma 3(a) and Lemma 3(b), according to a formula 
given in [12], Section 7.2. Finally the property X'g{0) = can be proved as follows. By 
deriving (18) (applied with fi = irg) at t = and by using the fact that ^ is centered, we 
haYcO = m,^,[Sn{ao)]^nyg{0) + Vg{0) + r'g jO). Hence A^(G) = 0. 

Remark 3. Notice that, according to (25)-(26), the constants Fg and Gg in Lemma 1 
also depend on the supremum in 9 of the norm of /.( in B' , , namely supggQ iJ.{V'^«). 



4. A Berry— Esseen theorem for Af- estimators 

Consider a Markov chain satisfying Assumption (A^) of Section 3.1. Let us introduce the 
statistic 

1 " 

M„(a):=-V^^(a,Xfe_i,Xfc), (27) 
n ^-^ 

k=l 

where a is the parameter of interest, F is a real- valued measurable function on A x 
and A is an open interval of the real line. 

Assume that F satisfies condition (Di) and let 

Mg{a):^ lim Ee.^[AU{a)] =Ee.^,[F{a,Xo,Xi)], 

n^oo 

which is well defined by Proposition 3. Assume also that, for each 9 G Q, there exists 
a unique ao = ao{9) G A, the so-called true value of the parameter of interest, such that 
Mg{a) > Mg{ao), \fa ag. To estimate ao = cko(^), we consider an M-estimator S„ as 
defined in Section 2, that is, M„(S„) < min^g^ Mn{a) + c„, where {c„}„>i is a sequence 
of non-negative real numbers going to zero. 

Let F' and F" be real- valued measurable functions defined on ^ x E'^ and let 

^ n 1 

M;(a):=- VF'(a,Xfe_i,Xfc), M;'(a) - V F"(a, Xfe). (28) 
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The functionals F' and F" could be the first- and second-order partial derivatives of F 
with respect to a, but this is not necessary to deduce our next result. Consider the 
following assumptions on F' and F" (and, imphcitly, on c„; see (V3)). 

Assumptions. 

(VO) F' and F" satisfy condition (D^). 

(VI) V0 e 0,Ee.X(,[-F"(aoj^Oj^i)] — and ao = olo{()) is unique with this property. 
(V2) m(6') :=Ee,^jF"(ao, ^0,^1)] satisfies inU^^emie) > 0. 

(V3) M'^iotn) satisfies condition (A3), that is, Vn > 1 and there exists r„ > indepen- 
dent of 9 such that r„ = o(l/-yn) anrf supggQ P6i,^{|M4(a„)| > r„} = 0(n~^/^). 

Notice that (VO) ensures supggg to(0) < 00 (see (17)). Now, as a consequence of Propo- 
sition 3 applied to F' and F" , the conditions (V0)-(V2) enable us to define the asymptotic 
variances: 



al{d) :=lim-Ee,^ 
n n 



al{0) :=lim-E, 



2n 



^F'(ao,^fc-i,^fc) 
L \fc=l / 

n 

Y,F"{a^,Xk^i,Xk)-nm{e) 



L \A:=1 



Moreover, condition (VO) and Proposition 3 ensure that supggQ(7j(0) < 00 for j = 1,2. 
The following conditions are also assumed to hold. 

(V4) infeeeaj(0)>O/or j = l,2. 

(V5) There exist rj G (0, 1/2) and C > such that 



\F"{a,x,y)~F"{a,x,y)\<C\a-a\iVix) + Viy)f. 



(V6) Set d := irdg^Qm{9)/8Tre{V'^') with 77 defined in (V5). There exists 7„ = o(l) 
such that 



supl 

See 



,{\a„ - ao\ >d}<j„ 



Theorem 3. Assume that Assumption (Ai) holds true, F satisfies condition {Di) and 
conditions (V0)-(V6) are fulfilled. Let t{9) := ai{9)/m{9). Then there exists a positive 
constant C such that 



Vn> 1 



sup sup 



'{9) 



(a„ - ao) < " ^ - r('u) 



<C 



1 



The statement in the above theorem corresponds to that of the i.i.d. case in [20] up 
to few changes: First, the variances of the i.i.d. context (namely, ¥,g[F' {9,XQf'] and 
¥.0[{F" {9 , Xq) — m{&)Y\ for an i.i.d. sequence {X„}„>o and a functional F{9,x)) are 
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replaced by the above asymptotic variances {8) and ct| {9) (this is natural in a general 
Markovian context); second, the uniform (in 6) third-order moment conditions (namely, 
snpg^QEe[\F'{e,Xo)\^ + \F"{e,Xo)\^] < oo) on both are replaced by the domina- 

tion condition (D3) for F' ,F"; third, even when F' = dF/da, here we allow for a positive 
sequence r„, n > 1, provided it decreases to zero sufficiently fast. The second point is spe- 
cific to the geometrically ergodic Markov chain case. Indeed, in the same statistical model, 
Dehay and Yao [5] proved a CLT for maximum likelihood estimates under a second-order 
domination assumption on the two first derivatives of the functional, which corresponds 
to inequality {D„io) with mo = 2. Here the previous second-order assumption is replaced 
by the (almost) optimal condition {D3) for deriving the Bcrry-Esseen theorem for M- 
estimators. 

Proof of Theorem 3. It suffices to check the conditions (Al)-(A6) of Theorem 1. The 
limit Mg{a) :— lim„E6/^^[A/4(a)] is well defined by Proposition 3 and condition (VO), the 
uniqueness of ao is guaranteed by (VI) and hence (Al) holds true. One more application 
of Proposition 3 ensures that Eg^Trg[F"{ao,Xo,Xi)] = lim„ Ee^p[M"(ao)], hence (A2) is 
satisfied. Condition (V3) is nothing else but (A3). The Berry-Esseen properties in (A4) 
are associated with the functionals F'{ao,x, y) and _F"(ao, x, y) respectively, so that they 
directly follow from Theorem 2. 

Now, let us check that (A5) holds true with a;„ = 0. Define W :— V^, where 77 € 
(0, 1/2) is the scalar in (V5) and notice that E^,^, [WiXo)^/"^] = ireiV). Next, since V > 1 
and 77 e (0, 1/2), we have 1<W <W^ <V so that 1 < 7rg{W) < MW^) < -^eiV) < h 
by property (VGl). Deduce that snpgf^Q ■JTg{W) < 00, and by Proposition 3 applied to 
C{e,x,y)^Wiy) 



Now, condition (A5) is guaranteed by the properties {A4) and (V5) with aj„ = 0, cw '■= 
supg^Q-KgiW) and Wn -.^ {l/n)Yl=i{W{Xk-i) + W {Xk)) provided that 



To prove (29), set 5„ := Y.l=i W{Xk). Since W„ < 2S'„/?^ + {W{Xo) + W{Xn))/n and 
^eA^MW) < Wn} < PeASn > 2n7rg{W)} + ¥eAW{Xa) + VK(X„) > 4mrg{W)} 




supPgA8MW) < Wn} = O(n-i). 



(29) 



< ^eASn - mrg{W) >n} + PgAW{Xo) + VF(X„) > An}. 
Equality (29) is then obtained by Markov's inequality. 
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since 

supsupE,.^[(W^(Xo) + < 2^'-^>-^[Tl{V) + Cui{V) + &i], 

6>Ge n>l 

using (a + 5)1/'' < 2i/''-i(ai/'' + for any a, & > and (VG1)-(VG2). Notice also that 
now condition (V6) is identical to condition (A6). 

The difficult part is to check the Berry-Esseen- type property (A4'). For this purpose, 
let 5 := {^,;(-, •, •), i G /} denote an arbitrary family of real- valued functionals defined on 
A X . Suppose that each is centered, that is, Eg.Trg [Ci(Q!o, -'^o, -'^i)] = for all i G / 
and e 0, and that condition (-D3) is fulfilled uniformly in i G /, that is, 

3m>3,3C>0,Vie/,Vae AV(2;,y)e£;2 \£^i{a,x,y)r <C{V{x) +V{y)). (30) 

For each i e /, set S'„(ao, i) '■— X]fe=i Cj(cto, ^fc-i, ^fe), and using Proposition 3, associate 
the corresponding asymptotic variance denoted by (yf\9). Moreover, assume that 



< inf{CT,;(6'), 61 e e, i e /} < sup{crj(6'), 6* e 6, i e /} < 00. 



(31) 



Then, we deduce from Theorem 2 that, under conditions {M.)., (30), (31) and ii{V) < 00, 
there exists a constant B such that 



Vn> 1 



sup sup sup 

iG/ eeOuGR 



'S»(q;o,0 



r(z.) 



< 



(32) 



This allows us to establish the two conditions in (A4'). Indeed, for (p, u) G N* x M with v 
such that \v\ < 2v'lnp, let us introduce the functional ^p_„ defined by 



^p,v{ao,x,y) := F'{ao,x,y) + 



V ai{0) 



Set Sn{ao,p,v) ■.= YJl^i^p,v{ao,Xk^i,Xk), and 



{F"{ao,x,y)-m{e)). 



ae{p,v) 



n 

^F'(ao,Xfc_i,Xfc), 



i{9), 



k=l 



SO that S„{ao,p,v) = S!^{ao) + ag{p,v)Sl^{ao). Notice that Eg_T,g[^p^y{ao, Xq, Xi)] = 
by (V1)-(V2). We have 

Ee,7re [Sn iao^P, v)"^] - ^e.Tve K,(ao)^] 

= aeip,vfEe^^,Kiao)^]+2aeip,v)Eg^^^[S'MSn{ao)]. 
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From (V2) and the fact that ai{-) is bounded, we have \ag{p,v)\ < A\v\/y/p for some 
^ > that does not depend on 9. Besides, as aheady mentioned in this section, one 
can define the asymptotic variances af{9) and cr|(6') associated with the functional F' 
and by 



1, 



a{{9) :=hni-Ee,7rJS','j(ao 



1 



a|(0):=hm-Ee,.«K:(ao)1- 



Similarly, the asymptotic variance cr^ ^{9) associated with ^p^t, can be defined by: 

Then it follows from |E0,,J5;(ao)5;'(ao)] I < Ee,., K(ao)']'/'IEe,.J5;'(ao)2]i/2 that 

WlM - '^!m < A'j<jI{9) + 2A^a,{9)a2{9). 

Since aj{-) is bounded (j = 1,2) and \v\ < 2-y/lnp < 2^^, the previous inequality shows 
that there exists C > 0, independent of 9, such that 



Set CTi := supggQ CTi(0) and ct^ := infg^Q ai{9) (we have oii > from (V4)). Using 
l^l/^/P — 2\/lnp/p and •^/inp/p = o(l), the above inequality implies that there exists 
-Po € N such that we have, for all p> Pq and v such that \v\ < 2^/lnp, 

V^ee iai<ap,„(0)<fai. 

In particular, under the same condition on (p, v), this gives ap^y{9) + ai(9) > 3ct]^/2, hence 
Wp,v{d) — cri{9)\ < 2C'\v\/3g_i^. This proves the first assertion in (A4'). 
Now, let us define 



I = {(j),v)eN* xR: p> Po>| < 2Vlnp}. 

It follows from (VO), (V2) and cti < +oo that the family E := {£,p^v,{p,v) & 1} satis- 
fies (30). Besides, the above bounds of (Jp^v{9) give the property (31). Then equation (32) 
shows that there exists B' > such that we have for all n > 1 , {p, v) E 1 , 9 E Q and u € M: 



Sniao,P,v) 
crp,v{9)y/n 



< u 



T{u) 



< 



B' 



Finally, let us fix any integer n> Pq and any real number u such that |u| < 2vum. Then, 
the previous Berry-Esseen bound with p := n and v :^ u provides the second property 
of (A4'). Indeed, we obtain from ^^(ao) = nM'^{ao) and ^^^(ao) = n{M^{ao) - m{9)) 
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that 



^ ^/;(ao) + ^J^(M;:(ao)-m(0))). 



Now the proof of Theorem 3 is complete. □ 

5. An example: AR(1) process with ARCH(l) errors 

Let us apply our theoretical results to an AR(1) process with ARCH(l) errors that 
belongs to the class of ARMA-GARCH models (see [7] and the references therein). The 
observations are generated by the process 

Xn = PoXn-i+o-{Xn-i;ao,bo)e„, n = l,2, (33) 

where Xq has some probability distribution /i, a'^{x; a, b) := a + bx'^ and \po\ < 1, qq, bo> 
are the true values of the parameters. {en}n>i is a sequence of i.i.d. random variables 
with zero mean and variance equal to 1, with finite pth order moment for some p to be 
specified below and (unknown) density that is continuous and positive on R. is 
independent of Xq. For simplicity, hereafter fj, is assumed to be the Dirac distribution Sq. 
The "true" parameter in the associated statistical model is the vector (po,ao,^o) € 
6 C [-p,p] X [ma, Ma] X [mb,Mb] C K^, where p € (0, 1), < ttIq < Ma < oo and < 
rrif, < Ml, < 1 are given such that {'p + ^/JIb)^ /^(l + \y\)^fe{y) dy < 1. For illustration, we 
apply our results to estimate pQ and ba. 

First, let us check that the Markov chain defined by (33) satisfies Assumption {Ai) of 
Section 3.1 with V{x) = (1 + \x\)p. To check (VG1)"(VG2) and the existence of the Qe- 
invariant probability measure tt^, by [17], Theorem 2.3, it suffices to prove that there exist 
constants g € (0,1), c, <;>0, a Borel subset S of the real line and a probability measure v 
concentrated on S such that the following two conditions hold true (see Remark 2): For 

aU flee, 

VxeM QeV{x)<gV{x)+c;ls{x) and Qe{x,-) > a^{-)ls{x). (34) 
In our setting, the transition probability of {X„}„>o is given by 

Qe{x, J ^b{pox + cr{x, oq, bo)y)fe{y) dy 
for any Borel set i? C M. As a consequence, for all 6* e 6 and a; G R, 
QeV{x) _ f V{pox + a{x,ao, bo)y) ^ {y)dy 



Vix) V{x) 

</( ^+^'-'+(-(^;^™ )\(.)d.. 
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By Fatou's lemma, 

limsupfsup^^;^) <(p+VA4r / {l + \y\rMy)dy^:i<l. 

Next, fix gi e (i, 1). There exists s > such that for each \x\ > s, QeV{x) < gV{x) for all 
9ee. Set S := [-s;s]. For all x € S and OeQ, 

QeVix)< [ {l+Tis + {^a + VM'bs)\y\ffM^V<^. 



so that the first condition in (34) is guaranteed. To check the second condition in (34), 
define 

Q<5{u):— inf f^{a~^[x,aQ,h()){u — p^x)), w G R. 
Then, for any xe Borel set i? C M and e 6, 

Qeix,B)^ / Isipox + cr{x,ao,bo)y)fi;{y)dy 
Js. 

fe{(T-^{x,ao,bo){u- pox))^^^ r 5{u) 



cr{x,ao,ba) Jb ma 

Define the measure m{du) m~^5{u) du and notice that m{S) > 0. We deduce from 
above that all 6* S 8, x G and Borcl set B C M, 

Qeix, B) > m{B) > m{B n 5) = 771(5)1^(5), 

where i' is the probability measure iy{B) := m{BC] S)/m{S). Hence the second condition 
in (34) is fulfilled and Assumption [M) is satisfied for {X„}„>o defined in (33). 
Second, to estimate pq, one can use the least-squares estimator, 

Pn ■■= — = argmm-2^F(p,Xfc_i,Xfe), 

l^k=l^k-l P ^ k=l 

where F{p,Xh^i,Xk) :~ {Xk — pXk-iY- We show that the assumptions of Theorem 3 
are satisfied so that we have a uniform Berry-Esseen bound for /)„. Fix some p > 6 
and recall that \yY'fe[y)dy < oo. Take F'{p,Xk-i,Xk) := -2Xk-i{Xk - pXk-i) and 
F"{p,Xk-i,Xk) ■.= 2Xl_^. The conditions (VO) and (VI) are obviously fulfilled. Next, 
define m(6') := Ee_^Ji^"(po,^fe-i,-'^fe)] and notice that m{e) / 2 = + [b^ + p}^m{e) / 2 . 
It follows that m{9) = 2ao/(l — Pq — bo) > 2ma and thus (V2) holds. Condition (V3) 
is satisfied with r„ = 0. From Proposition 3, we can use the Qg-invariant probability 
measure ttq to check condition (V4). Notice that lim„ E6/,,rs [-'^,^1 = m{6)/2 > nia and 
recall that {£«}„>! is i.i.d. We deduce that 

4 " 

al (9) = hm - V Ee.^, [Xl_y (Xk-i, ao, bo)el] > 4ao limEe.^, [Xf^] > Ami 

n Ti ' n 
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To derive a lower bound for cr| {0) , let us decompose 



•e,Trg 



.k=l 



fe=l 



E 



2 > Vk,, 

l<k<l<n 



where Vk,i ■.= Ee,^,[{F" (po, Xk^i, Xk) ~m{9)){F" {po,Xi^i, Xi)-m{9))], k < Lit is easily 
checked that Vk,i = (po + ^o)"*^?:,;-! for k < I. In particular, this implies Vk^i > 0, k <l. 
Next, by elementary inequalities, we can obtain infg E^.ttj, [(^"(poj -^^Oi -^i) — ^ ^ 

for some positive constant K depending on the variance of ef. Deduce that cr|(6') > 
hence (V4) holds true. Condition (V5) is trivially satisfied. To check the consistency of 
condition (V6), we take advantage of the explicit form of p„. Indeed, we have 



Pn - Po 



fc-i 



By Chebyshev's inequality, for any d > 0, Pe_^Q{|Ai„| > d} < d^^ri^^Eg^^^ [nA^„]. Propo- 
sition 3 guarantees that Eg^^,, [7iA^,j] is uniformly bounded (with respect to 0). Similar 
arguments apply to A2n- Since E^.Trg [^i] > rua for all 9, wc deduce that (V6) holds with 
7n = 0{n~^). Finally, by Theorem 3, there exists C > such that 



Vn> 1 



sup sup 

eeeueR 



ai{e)m{e) 



— {Pn~Pt))<u\-T{u) 



< (35) 



Third, let us now turn to the estimation of bo- For this purpose, assume that the e„'s 
have a moment of order p for some p > 12. Recall that ao = m(0)(l — Po ~ ^o)/2 and 
notice that Tq := m{9)/2 is easily estimated by := Sr=i -^k- Next, define 

1 " 

Tn{b;r,v) r]kib,r,vf 

k=\ 

with r^k{b,r,v) := (X^ - rXk^^f - v{\ - r^ - b) - bXl_^, 



with ^^ib;r,v) 
ob 



2 " d'^T 
- " Xl_^)r]k{b, r, v), -g^{b; r, v) 



k=l 



fc=i 



If Po a-nd ao were known, one could easily estimate bo by least squares, more precisely by 
minimizing T„(6;po,To) with respect to b. With this idea in mind, our feasible estimator 
of bo is defined as follows: 



: arg min Mn{b) 



with Mn{b) :=T, 



Define F'(&,Xfc_i,X,.):=2(T2-X2_i)77fe(&,po,T2),i^"(6,Xfe_i,Xfc):=2(r2-X|_i)2 and 
M;(&) ■=dTn/db{b]po,Tg), M'^{b) := d^Tn/db^{b] pq,t^). Let us point out that, in this 
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case, M^(-) and are only approximations of the derivatives of M„(-). Checking as- 

sumptions (V0)-(V2) is obvious and therefore we skip the details. To check condition (V3) 
for M^{bn), we use the decomposition A/,'j(6„) = An + A„ — An + Ai„ + A2„ + A^n with 

2 " 
n ^-^ 

k=l 

2 " . 

A„ := -^(tq^ -X^„i)(77fe(6„,po,To^) -?7fe(^i,P«,T,^)), 
fc=i 

Ai„ := — ^ y^('''o ^ ^k-i)i^k - PoXk-i)Xk-i, 
n ^ — ' 
fe=i 

A2n •= 2^(^0 " j^/c-l' 

fc=l 

A3„ 2{?l{l - pI -K) - tI{1 - pI - rl + XHn). 

We check that each term satisfies condition (V3) with a suitable r„. First, we can write 

= = A. + with i?„ := oi V77fc(6„,p„,f^). 

k=l 

By elementary algebra i?„ = 2(t^ — Tq + p'^^)X'^/n. Using the Berry-Esseen bound 
for (see Theorem 2) and Markov's inequality for X'^'^"' for some small a > 0, we 
can prove that ^e^AWn] > n^^} = 0{n-^^'^) so that Pe^^JIA,,] > n^i} = 0{n-'^^'^). By 
the bound in equation (35), we have supg Pe^SoUPn — PoP > n^-'/^ log-'^'^ n} ~ 0(n^^/^), 
j = 1, 2. Use this with j = 1 and our Theorem 2 for the centered functional ^(X^, = 
(r2 - Xl^^){Xk - poXk-i)Xk^i to deduce that P0^5„{|Ai„| > 7i-ilogn} = 0(n-i/2). 
Next, the bound on \pn — poP and Theorem 2 applied to the centered functional 
^{Xk, = (t2 - Xl_,)Xl_, - r4 +Ee,^, [X^.J allow us to deduce that Pe,5„{| A2„| > 

logn} = 0(n~^/^). Finally, use the Berry-Esseen bounds for p„ and and Markov's 
inequality for with some a > to deduce that Pe,5o{|A3„| > logn} = 0(n^^/^). 

Combining these facts gives that M^(6„) satisfies condition (V3) with r„ =ri~^logn. 
Condition (V4) can be checked using similar arguments to those used for jo„ and, there- 
fore, the details are omitted. Condition (V5) is trivially satisfied. Finally, let us note that 

r , _ ELi(^n--^fc-i)^fc(&o,P«,T^) 

On Oq - „ _ -v-2 N2 ' 

l^k=l\^7i ^k~l) 

and thus condition (V6) can be checked by arguments that we already used in this 
example. We deduce from Theorem 3 that, for some suitable t{9), 



Vn > 1 sup sup 



-^{K-ho)<u\-T{u) 



= 



los 
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The log factor in this Berry-Esseen bound is the price we pay for estimating 60 by a sim- 
ple two-step procedure, easy to implement, where we first estimate p„ and and then 
we use the least-squares criterion M„(6) = T„(6; p„,T^). We feel that the log factor could 
be removed by using a direct approach where the three parameters are estimated simulta- 
neously, but the investigation of this idea with Markov chain data is left for future work. 

6. Conclusion 

In this paper, we study the Berry-Esseen theorem for Af -estimators (or minimum con- 
trast estimators) of some parameter ao on the real line. The estimators are defined from 
a criterion based on a functional i^(Q;, X„_i, X„) of the observation process {X„}„>o. 
Our approach to derive such bounds relies on Pfanzagl's method originally proposed for 
i.i.d. observations [20]. In a first step. Theorem 1 in [20] is extended to obtain Berry- 
Esseen bounds for M-estimators based on any sequence of observations satisfying suitable 
conditions. In a second step, the specific case of ^-geometrically ergodic Markov observa- 
tions is considered. We show that such Markov framework allows us to apply our general 
result provided that F and related functionals F',F" satisfy suitable domination condi- 
tions. This result covers those reported in [19, 21], which are proved under much stronger 
moment conditions. We argue that the domination conditions used in the present paper 
give an almost optimal treatment of Berry-Esseen bounds for y-geometrically ergodic 
Markov chains. This is possible due to the operator-type procedure developed in [12]. 

There are several possible extensions of our results. A straightforward one is to follow 
the lines of the proof [20], Theorem 2, and to consider an estimator of the standard 
deviation in the Berry-Esseen bounds when this standard deviation depends on 9 only 
through ao - The details are omitted. Next, for more effective bounds, we need to carefully 
evaluate the constants involved throughout the paper. This is a direction of future work. 
Finally, there is no doubt that the operator-type procedure in [12] could be further used in 
statistical applications with Markov models, in particular with strongly ergodic Markov 
chains. This is under investigation. 

Appendix A: Complements for the proof 
of Theorem 1 

The reader is referred to Proposition 2 and its proof for the notation and the definitions 
used throughout this part. The following lemma gives key properties of the random 
functions g^ . 

Lemma A.l. The following properties hold true. 

1. If Vnfi ■= \/n{an- ao)/T{e), then A„ C {g^ {vum) <0 < 9~^{vn.e)} ■ 



2. For uj G Dn_0, g^ are increasing on the interval {—2\/mn, 2\/\nn) provided that 






(A.l) 
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Proof. We can write from assumptions (A5) and (A3) 

\nM^{ao) + (5„ - ao)nM'^{ao)\ = \nM'„{an) - (S„ - ao)ni?„(ao, S„)| 

< nr„ + n|S„ - ao||i?„(ao,S„)| 

< nr„ + 7i|S„ - ao|[|a„ - ao| + w„]W„. 

If w e An , then 

|nM,j(ao) + (S„ - ao)nM"(ao)| < n\a„ - ao\'^cw + nuJn\S„ - ao\cw + nrn. 
This last inequality is rewritten as 

n[M!^{ao) - r„] + t(6')\/"[M"("o) - sign(i/„.o)cH"^n]j^n,e - T{e)'^cwi^l,0 < 

and 

n[A/,'j(ao) + r„] + T{9)^/n[M''{ao) + sign(i/„^e)cvva;„]^'„,e + T(6')^CH'^'^,e > 0: 

with z/n^g := Y^(5!„ — ao)/T(0). Since < t{6) < a, we obtain that 

9~i'^n,e) < and > 0. 

The second statement is proved as follows for Note that a+ > and f/+ is con- 
tinuous. If we restrict v <0, the minimum of this quadratic function g'^{v) is achieved 
at 

_ b+ _ T(g)Vn[M4^(ao)-CH/oj„] 
2a+~ 2a2cv^ 

or at the origin if I'min ^ 0. Now, if w G Dn^e and n satisfies condition (A.l), it is easy to 
check that 

Vmin < -2Vlnn 

and is strictly increasing on (0,oo). Hence, 17+ is increasing on 2\/lnn). 
Similar arguments apply for . □ 

Lemma A. 2. We have for n large enough and \u\ < 2\/lnn 

K.e.u n Bn,e C Dn^g^u n B„,e C -B+g „ n B„,9. (A.2) 

Proof. It is understood below that uj G i?„,e- Since Bn^g C En.g H D„,6( and \u\ < 2-\/lnn, 
the second statement in Lemma A.l guarantees that for n large enough 

Vn{an~ao)/T{0)<u =^ .g+(V^(S„ - ao)/T(6')) < 5+(it). 

Since i?„,6/ C j4„, the first assertion in Lemma A.l yields g'^ {\/n{an — ao)/T{6)) > so 
that g^{u) > when ^/n{an — ao)/T{d) < u. This proves the second inclusion in (A.2). 
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Next, assume that g^{u) > 0. Since is increasing, we have 

y/n{a„~ aQ)/T{0) > u .g"(\/n(S„ - Q:o)/r(6')) > > 0. 

Since B„.e C A„ , we know from Lemma A.l that g^ {y/n{an — aQ)/T{9)) < which is in 
contradiction with the above inequahty. Thus, g~{u) > gives \/n(dtn — aQ)/T{9) < m. □ 



Appendix B: Complements for the proof of Lemma 1 

A first step to control the constants in Lemma 2 is to study the resolvent map (z — Qe)^^ 
of the transition kernel Qg acting on B~f. 

Lemma B.l. Let 6,r be such that < r < 1 and < 6 < 1 — r. Then, for any z e C 
such that |z| > r and |z — 1| > S, the operator z — Qg is invertible on By, and we have: 

HyiS,r) snp{\\{z - Qgy^\\-y,9 e 9, |z| > r, |z - 1| > (5} < oo. 

Proof. Let g g S^, and let us write kg = g — '!Tg(g)lE- Since T:g{hg) = 0, it follows from 
(VG2) that \\Qghg\\y < C^«;^||/ie||7- Now assume \z\ > r. Then 

\z\-(^^'^me\\, <9:LY^(<\'^\\hg\\, < -^\\hg\U. 
fe>0 ' k>0 ^ ^ ' 

Thus, tpg := X]fc>o z~^^^^''Q'ghg is absolutely convergent in B^, we have {z — Qg)'ipg = kg 
and |lV'6i|l7 < C!-y\\hg\\~^ / (r — k^). Besides, if z 7^ 1, then we clearly have 

Now assume \z\ > r and |z — 1| > S. Then the function fg := {■Kg{g)/{z — l))!^; + is 
such that {z~Qg)fg = g. Thus {z~Qg)~^g = fg. From (21), we obtain |7re((7)| < Tr^dgl) < 
7re(V^^)||g|l7 < h\\g\\y and \\hg\\y = \\g~ Mansh < (1 + bi)\\gh. This gives: \\fg\\y < 
{bi/S)\\g\\j + Cy{l + bi)\\g\\y/{r-Ky), hence Hy{S,r) < [bi/S + Cy{l + bi)/{r - k^)] <oo. 

□ 



Second, the constants involved in the Doeblin-Fortet inequality and the weak continu- 
ity condition of the Keller-Liverani theorem are proved to be uniform in 9 and to depend 
on ^ only via the constant of {Dmo). We appeal to [14], Remark, page 145, and to 
the improvements given in [15]. In the context of strongly ergodic Markov chains, the 
hypotheses resulting from [14, 15] are stated in [12], Section 4, and used here with the 
auxiliary norm sup on B^. In the sequel, for < 7 < 7' < 1, we denote by 

C{B~f,B~f') the space of the bounded linear operators from B-y to B-y', and by |( • W-y^-y' the 
associated operator norm (with the convention || • \\~f = || • when 7' = 7). 
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Lemma B.2. Let 7 <E (0, 1). We have: 

(a) ye G Q,yt e M,Vn > 1,V/ e B^, WQeitr/h < ^^7'^"ll/!l7 + ^ill/Ill/ 

(b) ye G e, e m, hq^W - Qe\U,i < 22-7q(i-^)/™(£;^ + E,)\t\^'^\\f\\^, 

where := sup^^Q £'1 := supggQ ||(5e|| 1 and C^, k^, bi are defined in (21) 

and (22). 

Proof. By using the inequality ||Q6i(i)"/ll7 < Il<3el/lll7i assertion (a) easily follows 
from (22) and (21). To establish (b), let us recall that we have from (Dmo) (use V >1) 

ie(e,x,y)|i-^ < cf-^)/"(v^(x) + y(j/))^-^ 

< 2i-^CJ'"'''/™(T/(a;)i-T + V{vf-i). 

Let / G S^,. From the definition of Qe{t)f and the inequalities \f\<V^\\f\\^, |e'" - 1| < 
2|a|^~^, we obtain that 

\{Q6{t)f){x) - {Qef){x)\ < 11/11^ / |e'*«("o,-.y) _ i\V{yyQeix,dy) 

JE 

< 22-''Q(i-^)/™|i|i-^||/|i^[y(x)i-^(geF'')(x) + {QeV){x)], 
from which we deduce (b) . □ 

For the next lemma (used to prove Lemma 3), we introduce the following notation. 
For any G 9, fc G N, i G K, let us denote by Qe,k{t) the operator associated with the 
kernel: Qe,fc(i)(x, dy) = i^C(ao, a;, y) V*«"°'-'^)Qe(':E, dy) (xeE). 

Lemma B.3. Let < 7 < 7' < 1 and k ~0, . . ., niQ : 

(a) // 7 + fc/m < 7' < 1, then the map t i-^ Qg kit) is continuous from ^ to C{B-y,Byi). 

(b) Ifk<mo — l and J + {k + l)/m < y < I, then the map ti-^ Qg^kit) is continuously 
differentiable from M to L{B~f,B~fi), and for all < G K, (AQg^kl 'dt)(t) is the operator in 
C{B^,B^>) associated to the kernel Qg^k+i{t)- 

Finally, we have Q^. ,y ,y, := sup{||Qe,fc(t)||^^^/,6' G 6,i G M} < 00, and Qk,^,-^' depends 
on ^ but only via the constant of (Dmo)- 

Proof. Set Ae.fc := Qg,k{t) — Qe,k{to), and let < e < 1 be such that 7+ {k + e)/m < 7'. 
Using |e'° - 1| < 2\a\^ and (-DmoX we obtain for / G B^: 

\Ag,kf{x)\<2\t-to\'\\f\uJ \ac^o,x,yt+^V{yyQg{x,dy) 

<2'+^'^+'^/'^C^^''+''>/"'\t-to\'\\fmV^''+''^/"'ix)QgV''{x) + QgV^'{x)). 

Since the functions V~^QgV^ and V^^ QgV^ are bounded on E uniformly in e G Q, we 
deduce that \\Ag^kf\\j' < l^~io|'^||/|l7, where is a positive constant depending on 
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(but independent of 9). This gives (a). The proof of (b) is similar, using the operators 
QeAi) - QeAto) -{t- to)QeM+iito) and the inequahty je'"^ - 1 - ia| < 2|a|i+^ □ 
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